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Abstract 

The ADVISOR (revised form) was used to obtain student ratings of Instruc- 
tors and courses for classes offered In spring 1971. The classes were divided 
according to nine groups according to the beginning time of class meeting In 
order to assess possible differential ratings by time. 

The first two subscales of the ADVISOR, Overall Evaluation of Course and 
Instructor » yielded significant overall F-ratios I'lom a one-way analysis of 
variance, but none of the individual comparisons among the nine class meeting 
times were significant. The other two subscales did not yield significant 
overall F-ratios. By utiliaing the above results and O)^ to assess the degrea 
of rolationship between ratings and time of class meeting, it was concluded 
that the time of day at which classes meet does not appreciably affect instruc- 
tor or course ratings. 



DOES THE TIME OF COURSE MEE^'ING ^ 
) AFFECT COURSE RATINGS BY STUDENTS? 

The use of student ratings to evaluate teaching at the college level is 
an important aspect within a general system of faculty evaluation. Input 
from student ratings has assumed greater prominence recently because of the 
desire to utilize the ratings in decisions involving the promotion and pay 
rate of faculty members. If the relative ranking of a faculty member obtained 
through an objectively-scored rating scale is important, then it is also impor- 
tant to investigate factors other than actual teaching ^fhich may affect faculty 
rankings. One such factor is the time of class meeting. 

A survey of the relevant literature was undertakc^n to determine if time 
of class meeting had been related to student ratings. No studies were found 
on this topic. Thus, the purpose of this study was to examine the extent to 
which time of class meeting would affect the course ratings by students. A 
priori, lower ratings might be expected for early morning classes, say eight 
o'clock, since these classes may be considered as unusually early by the 
students and faculty. Also, the classes meeting at the end of the day might 
expect to receive low ratings because of the exhaustion on the part of the 
students or instructor. 

Method 

Instrument 

The student rating form used in this study was the ADVISOR (revised form) 
which was developed in the fall of 1968 by students with the cooperation of the 
Measurement and Research Division, Office of Instructional Resources, University 
of Illinois. The -^VISOR consists of 34 objectively-scored items which are 
grouped into four subscores, plus open-ended questions on the reverse side. 
The four subscores are (1) Overall Evaluation of Course, (2) Instructor, 



(3) Quiz or Discussion, and (4) Laboratory or Language Laboratory. The number 
of items in each subscale are shown in Table 1. Feldman (1970) presented 
details of the development of the ADVISOR. For the complete form see Appendix A. 

Table 1 
Subscores of the ADVISOR 



Sub score 


No. of Items 


1. 


Cverall Evaluation of Course 


18 


2. 


Instructor 


6 


3. 


Quiz or Discussion 


6 


4. 


Laboratory or Language Lab. 


4 



The first and second' subscores apply to all courses. The remaining two 
subscores apply only to applicable course types. There are both positive and 
negative items in the ADVISOR. Each item has the Likert format with the 
following alternatives: Strongly Agree (SA), Agree (A), Disagree (D), and 
Strongly Disagree (SD) . These alternatives are coded 4, 3, 2, and 1, respec- 
tively, for the positive items axA the reverse order for the negative items. 
Each item has a pre-determined best answer which receives the highest rating* 
The last two items of the first subscale were excluded from the analysis since 
they were concerned with number of hours of homework and types of examinations 
given to the class. 
Data Sottroe 

The subjects used in this study were students who were enrolled in 
courses offered in the spring of 1971, at the University of Illinois, 
Champaign-Urbana campus, and whose instructor chose to use the ADVISOR form. 



3. 

Only section means (individual classes) were used in the analyses, and the 

number of sections is presented for each analysis performed. 

Procedure 

The data described above was divided into nine sets, each set correspond- 
ing to the begirming time of a class from 8:00 a.m. through 4:00 p.m. Classes 
beginning after 4:00 p.m. were excluded. No differentiation was made between 
classes with differing time lengths, that is, a class which met for two hours, 
say 8i0p a.m. to 10:00 a.m., was classified in the same set as an hour class 
which began at 8:00 a.m. 

The unit of analysis chosen was the section (individual class) mean. One« 
way analyses of variance were applied to the section means of each of the four 
subscores utilizing time of class meeting as the independent variable • 

The null hypothesis was that there was no significant difference kmong the 
ratings using time of class meeting as the basis for subject groupings. The 
probability level adopted for significance testing was .05. 

For any analysis that resulted in a significant F- ratio, Scheffe's Method 
(Ferguson, 1966, p. 296) of multiple comparison was chosen to test for individ- 
ual differences. In addition, the overall strength of association between the 
time of class meeting and the subscore ratings was estimated by usin^ the 
formula provided by Hays (1965, p. 382): / 

^ 2 S S betw een - (J - 1) MS within 

est as = ^ — 

SS total + MS within 

J = # of groups 

ResultQ 

Overall Evaluation of Course (16 Items) 

The means and standard deviations of the overall rating for the nine class 
meeting times are shown in Table 2. The range of means was from 3.17 for 4:00 p.m. 
classes to 2.91 for 11:00 a.m. classes. It may be important to note, however, 



that the mean for 11:00 a.m* was based on 120 sections, while the mean for 
4:00 p.m, was based on only 13 sections. ^ 

Table 2 

Overall Evaluation of Course 



Time of Class Meeting 


No. of Sections 


Mean 


Standard Deviation 


8:00 a.m. 


80 


3.02 


.27 


9:00 a.m. 


125 


2.98 


.34 


10:00 a.m. 


130 


3.04 


.30 


11:00 a.m. 


120 


2.91 


.33 


12:00 p.m. 


43 


3.02 


.27 


1:00 p.m. 


113 


2.9,8 


.42 ■ 


2:00 p.m. 


78 


2.98 


.50 


3:00 p.m; 


62 


3.06 


.35 


4:00 p.m. 


13 


3.17 


.37 



The results of the one-way analysis of variance, which was significant at 
the i05 level, are shown in Table 3. The index U)^ indicated that only about 
one percent of the variance in ratings was accounted for by t^e time of class 

meeting. Considering this result, it was not surprising to find that none of 

/ 

the individual comparisons (Scheffe) were signif ica:nt . 

Table 3 
Analysis of Variance 



Source 


df 


SS 


MS 


F 




Between 


8 


2.0185 


.2523 


1.99* 


.0102 


Within 


755 


9-5.9214 


.1270 






Total 


763 


97.9399 





*p < .Od 



Instructor (6 Items) 

The means and standard deviations of the instr'uctor ratings for different 
times of the day are presented in Table 4o As in the previous analysis, the 
instructors teaching classes at 4:00 p.m. received the highest rating, while 
those teaching classes at 11:00 a.m. received the lowest rating. 

Table 4 ' - 



Time of Class Meeting 


Sample Size 


Mean 


Standard Deviation 


8:00 a.m. 


80 ^ 


3.06 


.34 


9:00 a.m. 


125 


3.01 


.49 


10:00 a.m. 


130 


3.09 


.52 


11:00 a.m. 


120 


2.93 


.51 


12.00 p.m. 


43 


3.10 


.45 


1:00 p.m. 


113 


3.06 


.55 


2:00 p^m. 


78 


3.07 


.58 


3:00 p. in.. 


62 


3.17 


.44 


4:00 p.m. 


13 


3.31 


.38 



The F-ratio from the one-way analysis of variance, shown in Table 5, was 
significant at the .05 level. The hypothesis of no effect of the time of class 
meevting on the ratings was rejected. No pair of the Scheffe individual com- 
parisons were found to be significant. Only 1.17 percent of the variance in 
rating was accounted for by the time of class meeting as indicated by (si . 
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Table 5 ^ 
Analysis of Variance 



1 

Source 


df 


SS 


MS 


F 


'2 


Betv7een 


8 


4.2132 


.5267 


2.13* 


.0117 


Within 


755 


187.0215 








Total 


763 


191.2347 





p < .05 



Qu%z or Diccvssicn (6 Items) 

Means and standard deviations of the Quiz or Discussion subscore ratings 
for all groups are shown in Table 5. The 12:00 p.m. classes received the 
highest rating, and the lowest rating was for 8:00 a.m, classes. The F-ratio 
was not significant (see Table 7). Only 1.15 percent of the variance in ratings 
was indicated by 0)^ to be .^ittributable to the time of the class meeting. 

Table 6 
Quiz or Discussion 



ERLC^ 



Time of Class Meeting 


Sample Size 


Mean 


Standard Deviation 


8:00 a.m. 


65 


3.00 


.52 


9:00 a.m. 


99 


3.10 


.51 


10:00 a.m. 


97 


3.14 


.48 


11:00 a.m. 


104 


3.06 


.49 


12:00 p,m. 


38 


3.28 


.43 


1:00 p.m. 


92 


3.11 


.50 


2:00 p.m. 


68 


3.20 


.36 


3:00 p.m. 


49 


3.25 


.45 


4: 00 p.m. 


11 


3;27 


.53 



Table 7 
Analysis of Variance 



Source 


df 


SS 


MS 


F 




Between 


u 


3.5024 


.4378 


1.91 


.0115 


Within 


614 


140.7664 


.2293 






Total 


622 


144.2688 





Laboratory or Language Laboratory (4 Items) 

The means and standard deviations of the Laboratory or Language Laboratory 
are presented in Table 8. The classes meeting at 3:00 p.m. received the lowest 
rating (2.58), while the 8:00 a.m. classes had the highest rating (3.03). The 
F-ratio (Table 9) was not significant. Slightly less than one percent of the 
variance in ratings was accounted for by the time of class meeting as indicated 
by OJ^. 

Table 8 

Laboratory or Language Laboratory 



Time of Class Meeting 


Sample Size 


Mean 


Standard Deviation 


8:00 a.m. 


27 


3.03 


.44 


9:00 a.m. 


34 


2.64 


.69 


10:00 a.m. 


40 


2.81 


.73 

1 


11:00 a.m. 


28 


2.60 


.62 


' 12:00 p.m. 


11 


2.79 


.94 ' 


1:00 p.m. 


32 , 


2.61 


.72 


2:00 p.m. 


16 


2.86 


.66 


3:00 p.m. 


14 


2.58 


.72 


4:00 p.m. 


3 


2.83 


.38 
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Table 9 
Analysis of Variance 



Source 


df 


SS 


MS 


F 




Between 


8 


4.6753 


.5844 


1.26 


.0099 


Within 


199 


92.2863 


.4638 






Total 


207 


96.9616 





Discussion and Conclusion 

The one-way analysis of variance F-ratios for the Overall Evaluation of 
Course and the Instructor subscores were 1.99 and 2,13, respectively. The null 
hypotheses of no differences of ratings among groups in time effects were 
rejected at the .05 level of significance. The contention that the students 
rate classes and instructors differently according to the time of class meeting 
appeared to be supported by this statistic. 

However, these ratios were only slightly higher than the critical F-value, 
No individual comparisons among class meeting times were found to be signifi- 
cant, including the tests between highest and lowest mean ratings. Furthermore, 
the strength of association (estimated via 03^) between the time of class meeting 
and the ratings was small, i,e., only ,010 and .012 for the Ov<3rall Evaluation 
of Course and for the Instructor scales, respectively. Thus, the differences 
among classes meeting at various times, while statistically significant, has 
very little practical .significance*^ The statistical significances were probably 
due. to the large sample sizas* This conclusion is further substantiated by the 
fact that the two subscales with smaller sample sizes showed nearly equivalent 
strengths of association, but thoy were hot statistically significant. 

Since the data used in this study was not random, the observed results are 
not necessarily generalizable to other comparable situations. To some extent^, 



9. 



the students and the instructors both selected the time or the class meeting 
according to their preference • This characteristic may affect the ratings. 
If both teachers and students were randomly assigned to the classes and to 
the time of class meeting, the results might have been different from those 
obtained here. However, practically speaking, the time of day at which the 
class meets does not seem to appreciably affect the ratings. 
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MARK 
MARK 
MARK 
MARK 



iF YOU STRONG!. V AG RET 
WITH THE ITEM 

If" '^^^ AGREE MOD ERATCLY 
WITH the: I TtM 

IF YOU DlSAGRLE MODEnATELY 
WITH THE ITEM 

IF YOU ST RQNGL Y Ol SAGRUE 
■.VI TH THE I TEM 



OMIT ITEMS WHICH DO NOT APPLY TO THIS COURSE 
OVHRALL EVALUATION OF COURSE (These items relate only to the Lecture section.) 
I learned a great deal in this course. 
; ■ = ■■ • 1 enjoyed the course. 

; . ; . Too much irrelevant material was presented. 

I Examinations mainly tested trivia, 

i Grading was based on clear standards. 

The amount of jwork required for this course was excessive. 
['.■■■ Content of exarviinations was unfair. 

,;, This course has foggy objectives. 

v; ' . Out -of - class work (i.e. Homework) was relevant and helpful 

. ' Grading in this ooufse has been unfair. 

' ■ : ; ■ The course material seemed worthwhile. 

; AN that fs required for th(S course is inenjorfzation. 
^ . V . : : Content of the course was good. 
' ' '., This course w-.s a waste ol time. 

Note taking in the lecture was difficult. 
; This course could be considerably improved. 

J Considering all homework (i.e. papers, projects, reading), how many hours per week did you spend studying? 
: 1^"" ■"]' r ^^'^CA. 0-3, B. 4-6, C. 7-9, D. 10 or more) 

; What types of exams were given? 
: : ' ; (A. Objective or mathematical problem solving rnly, 

INSTRyCXQR {These items relate only to the n^ain instructoj.) 

Wiite your main instructor's name. / _ . , 

•• : ' : ; The presentations by the instructor (s) were excellent. 
; ; - I ■ : ^ ' The instructor seems to dislike teaching this course. 
■ ; ■■ The instructor was interesting. 

• -| • The personality of the instructor was a problem. 

. ; [ .: \ I would like to take another course taught by this instructor, 
'i i ■ ■ • i In general, | rank this instructor higher than others 1 have had. 



I 



B. Essay only, C. Both A. and B.) 



QUIZ OR DJSCUSSION (These items relate only to the Quiz or Discussion section.) 
\"lntc yoiif Quiz oj n_ismsioa iastruclor's name (T.A.) ; 

; .-i : ; This c! . :s helped to clarify texts and lectures. 

I ; ■ ' l - 1 There was ample opportunity to ask questions. 

* ; ;■■ I •.-•'I . i The T.A. did not cover enough of the subject matter introduced in lectures. 

I ' A ; .: : -, j This T.A. is an effective instructor. 



j Questions were answered clearly and appropriately. 



i - i: ] This class was valuable to the understanding of the course. 



i LABOMIORY.0R^.LMGy^ (These items relate only to the Laboratory or Language Lab.) 

; Wiite youf Laboraiory instructor's name „.. . 

; " ]Tl i SATT[lJ|scr| Ttie lab assistant was very helpful. 
) ' ■ n ■ ^ Thprp werp too manv oroblems v^ith 1 
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There were too many problems v^ith the lab equipment. 
Too much time was wasted in the lab. 



j y,^ ^^[■'^j ^ work was weii organized. 
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n,r ,r , r . THE ADVISOR , , , . p . 

OtfiCKii Lvaluation Porm of 1 eacher - Course Iiv;jlu£it3on Guide 

Champ'cHgn - Urbani^ Cf^mpus University of Illinois; 

, Side One ' 

Please use this side of the form for your personal comments on teacher effectiveness and general course value--then turn it over and 
answer the objective questions on the other side, using pencil oniy . Refneniber - -these questionnaires will be collecled by a sluder^l 
in your class and niailed by hjni^to the Advisor staff. Your Instructor will not see your completed evaluation. 

COURSE CONTENT 

Please gtve youf cof;;ments on the coufse contenl, subject ;v;atler and any oailicuiar [!?leva!ice ihis cji;[GC tias had lo yoiif 3ioa of study. 



INSTRUCTORS Write the name of youi Principle instructor < T.A, 

What are your genera! comments about the instiiictor in this course? 



PAPERS AMD HOMEWORK 

Coninienl on the value of books, homework, and papers (if any) in this course. 



EXAMS 

Comment on Ihe exams (quizzes, practicals) as to difficulty, fairness, etc. 



GENERAL 

1. What improvements in this course would you suggest? 



2. Please give your thoughtful evaluation of this course with comments. Are you satisfied with what you got out of this course? 
Do you considei it a valuable educational experrence^ Simply a means of passing a requirement? Oi a disappointment? 
Please comment. 
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